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Abstract 

We give constructions of n'^ x n'^ x n tensors of rank at least 2n'^ — 
0(n'°~^). As a corollary we obtain an [nY shaped tensor with rank at 
least 2nL'"''^J — 0(nL''/2J-i) when r is odd. The tensors are constructed 
from a simple recursive pattern, and the lower bounds are proven using 
a partitioning theorem developed by Brockett and Dobkin. These two 
bounds are improvements over the previous best-known explicit tensors 
that had ranks n* and n^^^^^ respectively. 
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1 Introduction 



An important and well-studied property of linear operators, equivalently matri- 
ces, is their rank. Much is understood about the rank of matrices over fields, 
and an efficient algorithm exists for the calculation of the rank of an explicit 
matrix. However, a closely related problem, calculating the rank of a tensor, a 
generalized version of a matrix, has been shown to be NP-complete[l], and so is 
unlikely to have an efficient algorithm. Due to the intractability of the problem, 
very few results have been shown on this subject. 

1.1 Importance of Tensor Rank 

The rank of a tensor is relevant and important in several different settings. 
Fast matrix multiplication, a problem that is of incredible importance, can be 
improved by improving the upper bound on the rank of a related tensor. A 
recent paper by Ran Raz proved two theorems relating lower bounds on the 
rank of tensors and lower bounds on the size of arithmetic formulas: 

• Theorem: Let A : [nY W be a tensor such that r < 0(logn/ log log n). 
// there exists a formula of size n'^ for the polynomial 

r 

fA{xi,l,...,Xr,n)^ ^ A{ii, . . . , ir) ■ Y[ ^ 3,tj 
ii,...ire[n] j=l 

then the tensor rank of A is at most n'" *^^"^ ' '^[H]- 

• Corollary: Let A : [n]^ be a tensor such that r < 0(logn/ loglogn). 
// the tensor rank of A is > n^'(^~°(^'>'> then there is no polynomial size 
formula for the polynomial fA [2] ■ 

These two theorems give a strong motivation behind finding explicit [nY tensors 
of high rank. In this paper we give an explicit hypercube tensor with rank 
approaching 2nL''/^J , an improvement over the previous best-known example by 
a constant factor of 2. 

1.2 Methodology 

For each integer fc, we will give an x x n tensor with rank at least 2n^ — 
0{n^~^ ) , an improvement over the previous best-known n'^ . To do so, we will use 
a partitioning theorem developed by Brockett and Dobkin in [3] . This theorem 
allows us to lower bound the rank of a tensor that is formed by concatenating or 
gluing together other tensors, provided they are sufficiently different. We will 
construct a tensor recursively by continually gluing together three copies of a 
smaller tensor. The partitioning theorem will allow us to lower bound the rank 
of the tensor at each step, and thus the final tensor as well. As a corollary we 
will construct an n x • • • x n tensor of rank 2nL'"/^J — 0(nL''/^J~^) when r is odd 

r times 

by viewing the first construction under an isomorphism. This is an improvement 
over the previous best-known nL»'/2j \yy g, constant factor. 
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1.3 Definitions 



Throughout this paper, F will denote a field. Let A S F»iX"2xn3^ jf 

A = Xi 1^ X2 <^ X3 

for Xi G i^"' and A is nonzero, then A is called a simple or rank-1 tensor. The 
rank of a general tensor A is defined as the minimal number r such that we can 
write 

A = (1) 

1=1 

where each Bi is a simple tensor. This is a natural extension of matrix rank, 
because if 0:3 = 1 then the rank of A agrees with the matrix rank. Throughout 
this paper, R[A] will denote the rank of A. 



1.3.1 Slices, Concatenations, and the Characteristic Matrix 

Let A e i^"i^"2xn3 fix a positive integer 1 < fc < and let B g F"!^"^^! 
satisfy 

Then B is called the fcth slice of A. We will denote the fcth slice of a tensor A as 
Ak. The concatenation of tensors A e F"!''"^^™ and B e F"ix«2xm'^ denoted 
AB e F"ixn2x(m+m')^ jg tcnsor such that 

^ f Aijfe if 1 < fc < TO 

yfe if m < fc < m + to' 

The m + m' slices of the concatenation are the to slices of A followed by the m' 
slices of B. Also AB and differ only by permutations of the indices in the 
third dimension, so R[AB] = R[BA]. 

The characteristic matrix of A G F'"!^"^^"^ is a matrix A{s) with indeter- 
minants, i.e. A{s) £ F"!''"^ where E = FU S and 5" = {sj"|^ is a set of 
indeterminates with 

"3 

A{s) = ^s,A, 
1=1 

so each indeterminate represents the values on a different slice. Define dim s = 
\S\ = Us- Define the column (resp. row) rank to be the maximal number 
of linearly independent columns (resp. rows) as in [3]. Note that the row 
and column rank are not necessarily equal, for example A{s) = [si 82] has 
column rank= 2 and row rank= 1. We also sometimes write i?[A(s)] for R[A]. 
To avoid trivialities, we usually work with nondegenerate tensors; a tensor A 
is nondegenerate if no nontrivial linear combination of its slices vanish and 
its characteristic matrix A(s) has full row and column rank. An analogy of 
concatenation can be defined with characteristic matrices as well. Let A(s) and 
B{t) be two characteristic matrices of the same dimensions, and let dims = n 
and dimt = to. We define C{u) = A{s) + B{t) as 

n m 

i=i j=i 
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where u — sUt. Note C{u) is the characteristic matrix of AB, so this addition 
can be considered as a concatenation. 



1.4 The Partitioning Theorem 



The main tool we use in our construction is the partitioning theorem developed 
by Brockett and Dobkin in [3], and we write it here for easy referral: 

Theorem 1. Let G(s) be a nondegenerate characteristic matrix, and let one of 
the following cases hold: 



(i) G{s) 



Gi{s) 
G2{s) 



(ii) Gis) = [ Gi(s) G2(s) ] 

(Hi) Gis) ^ Gi{u) + G2{v) 
Then for each case we have 
(i) R[G{s)] > miuM R[Gi{s) + iVG2(s)] + row rank G2(s) 

(ii) R[G{s)] > minjvi?[Gi(s) + G2(s)M] + column rank G2(s) 



(Hi) R[G{s)] > minr R[Gi{u) + G2(ru)] + dimw 

for matrices M , N , and T sized so that the two summands are the same shape 
and the addition is well-defined, and juxtaposition means regular matrix multi- 
plication. 

This theorem essentially states that if two halves of a tensor "don't overlap 
too much", then each slice of the second half must add at least one to the rank. 
A special case of "don't overlap too much" is given in the following theorem: 

Theorem 2. Let Gi(s), G2{s), and G^{s) all be nondegenerate characteristic 
matrices. Then we have 



(i) R 



Gi(s) 
G2{s) G3(s) 
row rank Gi(s)} 



> max{i?[Gi(s)] + column rank G^(s), R[Gz{s)] + 



(ii) R[ Gi{s)+G2{t) Gsit) ] > m'dx{R[Gi{s)]-{' column rank Gsit), R[G3{t)]- 
dim s} 



(Hi) R 



Gi{s) + G2{t) 



> max{R[Gi{s)] + row rank Gsit), R[G 3 {t)]-i' dim s} 
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2 The Main Result 



In this section we give a construction that yields y. n tensors of rank 

approaching 2n^ . These numbers are, to the best of our knowledge, the largest 
known rank of any explicit tensor of these shapes. As a corollary, for r odd, 
these constructions allow us to give an [nY shaped tensor of rank approaching 
2nL''/2J J another improvement to the best of our knowledge. The first step is to 
prove a lower bound for a block tensor: 

Theorem 3. Let A G F™^"^^ he nondegenerate, B e pmxn'xp' nondegen- 
erate, and C £ prn'xnxp' nondegenerate and let E e pmxnxp' ^ q ^g 

the tensor of zeroes of appropriate dimensions to be concatenated, and let 



M 



AE 
OC 



OB 
00 



then 



R[M] > R[A] + column rank B{t) + row rank C{t) 
Proof. First, transforming into characteristic matrices. 



M{u) = 
with u ^ sUt. We partition 

M{u 

with 



A{s)+E{t) 
C{t) 



Bit) 




Giiu) = [ A{s) + E{t) 
G2{u) = [ C{t) ] 



Bit) 



By Theorem [U 



R[Miu)] > niini?[Gi(M) + NG2iu)] + row rank G2(m) 

By Theorem m 

i?[Gi(u) + NG2iu)] = R [Ais) + iE + NC)it) Bit)] 

> max{R[Ais)] + column rank + dim s} 

Since row rank G2{u) = row rank G(s), we have 

R[Miu)] > row rank G(s) + i?[A(s)] + column rank B{t). 

□ 

This theorem is the key to our construction. We recursively build a tensor 
as follows: pick a positive integer k and let A^^^ = Ink-^, and define 



A^O 0A(*' 
Oyl(*) 00 



the main result is 
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Theorem 4. Pick I = logn. Then the tensor A^''^ above has dimensions x 
X n and satisfies R[A'^^'>] > 2n'' - 0{n''^^). 

Proof. For any i, it is clear that A^"^^ is a 2^n''~-^ x x 2' tensor. Further- 

more, an easy induction shows that A^"^^ (s) is nondegenerate by noting that A^*^ 
always has at least one slice with full row and column rank, and a nontrivial 
linear combination of slices of A^^^^ that vanish is such a combination of slices of 
as well. Thus 



row rank A*-*-* (s) = column rank A^*' (s) — 2'r 
and A^'^ is nondegenerate. By Theorem [31 

> i?[yl(*)]+row rank A**) (s)+column rank A'^'\s) = R[A'^'^]+2'+^n''-'^ 
Then a straightforward induction shows 



i-l 



= n*--! + 2(2* - l)n''-^ 

setting / = logn, we have > 2n'' - n'^-i and A, e i^"'x«'x". □ 

This construction allows us to improve on the previous best-known explicit 
hypercube tensor by taking the preimage of these tensors under the canonical 
isomorphism. 

Corollary 5. Let r be odd, k = [r/2\, A^'-* as above, and let (j) be the canonical 
isomorphism 



pn X ■ ■ ■ X n ^ pn'' xn'' xn 



Xi (X) ■ • ■ (g) a;.r ^ (a^l 'X' ■ • ■ ® Xk) ^ (Xk+l ® • ■ • C?) X2k) 'Sy> Xr 

then is ann X ■ ■ ■ X n tensor with rank at least 2nL'"/^J — 0(nL''/^J^^). 



Proof. We show that for any tensor 5 g i?"- x • • • x i?[0(-B)] < R[B]. Assume 
the opposite towards a contradiction. Then if 

R[B] 
i=l 

for simple tensors Di, we have 

R[B] 

<t>{B) = J2 <t>{Di) 

i=l 

and as Di is simple, so is (j){Di), but since > R[B] this contradicts 

minimality of R[(f){B)], thus < R[B], so clearly < 

□ 

To our knowledge, these are the best-known ranks for explicit [nY and n'^ x 
n'' X n tensors for any k, including the important cube n x n x n tensors. 
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3 Conclusion 



In this paper we have presented an improvement to about 2n'' from the previous 
highest rank expHcit tensors for the n'' x n'^ x n shape. This extends to an 
improvement for the shape [nY when r is odd. These tensors were constructed 
by using Brockett and Dobkin's partitioning theorem in a recursive manner. 
However, using this theorem imposes a restriction on the quahty of the lower 
bounds. In order to improve further, we need to either improve the partitioning 
theorem or develop a different method. 



3.1 Open Problems 

• The most important open problem is the one presented as the motivation 
for this paper. The improvements in this paper do not come anywhere 
close to the n^'-^^°'-^'>^ threshold for hypercube tensors. Any explicit ten- 
sor with this rank would imply super-polynomial lower bounds on certain 
functions as per Ran Raz's recent theorem[5]. Any attempt to develop 
examples of high-rank tensors should keep this goal in mind. 



• An improvement to Brockett and Dobkin's partitioning theorem would 
be extremely useful. The same techniques presented here would be more 
powerful and perhaps improve by an increase in the exponent, rather than 
a constant factor. 



3.2 Additional Notes 

This paper is the result of research done at Caltech from June 2010 to August 
2010 as part of the SURF program. I worked under Chris Umans, Professor of 
Computer Science, and I'd like to thank him for all his help and advice while 
working on this project. Additionally, in between the writing and the submission 
of this article, an independent article was published by Boris Alexeev, Michael 
Forbes, and Jacob Tsimerman[T] that gives, among other things, an explicit n*^ x 
n'^xn {0, l}-tensor with rank at least 2n''+n—Q{k\ogn). The techniques in this 
paper are similar to those described here, so the two bounds are very close, but 
the one given by Alexeev, Forbes, and Tsimerman has better lower-order terms. 



Interested parties can read the paper here http : //arxiv . org/abs/1 102 . 0072 
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